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Abstract. Differential privacy is a notion of privacy that has become very pop- 
ular in the database community. Roughly, the idea is that a randomized query 
mechanism provides sufficient privacy protection if the ratio between the prob- 
abilities that two adjacent datasets give the same answer is bound by e*^. In the 
field of information flow there is a similar concern for controlling information 
leakage, i.e. limiting the possibility of infening the secret infonnation from the 
observables. In recent years, researchers have proposed to quantify the leakage 
in terms of min-entropy leakage, a concept strictly related to the Bayes risk. In 
this paper, we show how to model the query system in terms of an information- 
theoretic channel, and we compare the notion of differential privacy with that of 
min-entropy leakage. We show that differential privacy implies a bound on the 
min-entropy leakage, but not vice- versa. Furthermore, we show that our bound 
is tight. Then, we consider the utility of the randomization mechanism, which 
represents how close the randomized answers are to the real ones, in average. 
We show that the notion of differential privacy implies a bound on utility, also 
tight, and we propose a method that under certain conditions builds an optimal 
randomization mechanism, i.e. a mechanism which provides the best utility while 
guaranteeing e-differential privacy. 



1 Introduction 

The area of statistical databases has been one of the first communities to consider the 
issues related to the protection of information. Already some decades ago, Dalenius [1] 
proposed a famous "ad omnia" privacy desideratum: nothing about an individual should 
be leamable from the database that could not be learned without access to the database. 



Differential privacy. Dalenius' property is too strong to be useful in practice: it has been 
shown by Dwork [2] that no useful database can provide it. In replacement, Dwork has 
proposed the notion of differential privacy, which has had an extraordinary impact in 
the community. Intuitively, such notion is based on the idea that the presence or the 
absence of an individual in the database, or its particular value, should not affect in a 
significant way the probability of obtaining a certain answer for a given query [2-5]. 
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Note that one of the important characteristics of differential privacy is that it abstracts 
away from the attacker's auxihary information. The attacker might possess information 
about the database from external means, which could allow him to infer an individual's 
secret. Differential privacy ensures that no extra information can be obtained because 
of the individual's presence (or its particular value) in the database. 

Dwork has also studied a technique to create an e-differential private mechanism 
from an arbitrary numerical query. This is achieved by adding random noise to the result 
of the query, drawn from a Laplacian distribution with variance depending on e and the 
query's sensitivity, i.e. the maximal difference of the query between any neighbour 
databases [4]. 



Quantitative information flow. The problem of preventing the leakage of secret in- 
formation has been a pressing concern also in the area of software systems, and has 
motivated a very active line of research called secure information flow. In this field, 
similarly to the case of privacy, the goal at the beginning was ambitious: to ensure 
non-interference, which means complete lack of leakage. But, as for Dalenius' notion 
of privacy, non-interference is too strong for being obtained in practice, and the com- 
munity has started exploring weaker notions. Some of the most popular approaches 
are quantitative; they do not provide a yes-or-no answer but instead try to quantify the 
amount of leakage using techniques from information theory. See for instance [6-12]. 

The various approaches in the literature mainly differ on the underlying notion of 
entropy. Each entropy is related to the type of attacker we want to model, and to the way 
we measure its success (see [9] for an illuminating discussion of this relation). The most 
widely used is Shannon entropy [13], which models an adversary trying to find out the 
secret x by asking questions of the form "does x belong to a set ST\ Shannon entropy is 
precisely the average number of questions necessary to find out the exact value of x with 
an optimal strategy (i.e. an optimal choice of the S"s). The other most popular notion 
of entropy in this area is the min-entropy, proposed by Renyi [14]. The corresponding 
notion of attack is a single try of the form "is x equal to value vT\ Min-entropy is 
precisely the logarithm of the probability of guessing the true value with the optimal 
strategy, which consists, of course, in selecting the v with the highest probability. It is 
worth noting that the conditional min-entropy, representing the a posteriori probability 
of success, is the converse of the Bayes risk [15]. Approaches based on min-entropy 
include [ 12, 16] while the Bayes risk has been used as a measure of information leakage 
in [17, 18]. 

In this paper, we focus on the approach based on min-entropy. As it is typical in the 
areas of both quantitative information flow and differential privacy [19,20], we model 
the attacker's side information as a prior distribution on the set of all databases. In our 
results we abstract from the side information in the sense that we prove them for all 
prior distributions. Note that an interesting property of min-entropy leakage is that it is 
maximized in the case of a uniform prior [12, 16]. The intuition behind this is that the 
leakage is maximized when the attacker's initial uncertainty is high, so there is a lot to 
be learned. The more information the attacker has to begin with, the less it remains to 
be leaked. 
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Goal of the paper The first goal of this paper is to explore the relation between dif- 
ferential privacy and quantitative information flow. First, we address the problem of 
characterizing the protection that differential privacy provides with respect to informa- 
tion leakage. Then, we consider the problem of the utility, that is the relation between 
the reported answer and the true answer Clearly, a purely random result is useless, the 
reported answer is useful only if it provides information about the real one. It is there- 
fore interesting to quantify the utility of the system and explore ways to improve it 
while preserving privacy. We attack this problem by considering the possible structure 
that the query induces on the true answers. 

Contribution. The main contributions of this paper are the following: 

- We propose an information-theoretic framework to reason about both information 
leakage and utility. 

- We prove that e-differential privacy implies a bound on the information leakage. 
The bound is tight and holds for all prior distributions. 

- We prove that e-differential privacy implies a bound on the utility. We prove that, 
under certain conditions, the bound is tight and holds for all prior distributions. 

- We identify a method that, under certain conditions, constructs the randomization 
mechanisms which maximizes utility while providing e-differential privacy. 

Plan of the paper The next section introduces some necessary background notions. 
Section 3 proposes an information-theoretic view of the database query systems, and of 
its decomposition in terms of the query and of the randomization mechanisms. Section 
4 shows that differential privacy implies a bound on the min-entropy leakage, and that 
the bound is tight. Section 5 shows that differential privacy implies a bound on the 
utility, and that under certain conditions the bound is tight. Furthermore it shows how 
to construct an optimal randomization mechanism. Section 6 discusses related work, 
and Section 7 concludes. The proofs of the results are in the appendix. 

2 Background 

This section recalls some basic notions on differential privacy and information theory. 
2.1 Differential privacy 

The idea of differential privacy is that a randomized query provides sufficient privacy 
protection if two databases differing on a single row produce an answer with similar 
probabilities, i.e. probabiUties whose ratio is bounded by for a given e > 0. More 
precisely: 

Definition 1 ([4]). A randomized function JC satisfies e-differential privacy if for all of 
data sets D' and D" differing on at most one row, and all S C Range{IC), 

Pr[IC{D') eS]<e' X Pr[K:{D") e S] (1) 
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2.2 Information theory and interpretation in terms of attacks 

In the following, X, Y denote two discrete random variables with carriers X ^ {xq, . . . , 
Xn-i}, y = {yo, ■ ■ ■ , Um-i}, and probability distributions px{'), Py{'), respectively. 
An information-theoretic channel is constituted of an input X, an output Y, and the 
matrix of conditional probabilities py\x{' I •)' where PY\x{y I represent the prob- 
ability that Y is y given that X is x. We shall omit the subscripts on the probabiUties 
when they are clear from the context. 

Min-entropy. In [14], Renyi introduced a one-parameter family of entropy measures, 
intended as a generalization of Shannon entropy. The Renyi entropy of order a (a > 0, 
a 7^ 1) of a random variable X is defined as Ha{X) — jz^^og2j2xexPi^)°'- 
We are particularly interested in the limit of Ha as a approaches oo. This is called m/n- 

enfropy. It can be proven that i/oo(^) ^^'nia^oo Ha{X) = — log2 max^^g^:' 

Renyi also defined the a-generalization of other information-theoretic notions, like 
the Kullback-Leibler divergence. However, he did not define the a-generalization of 
the conditional entropy, and there is no agreement on what it should be. For the case 
a = 00, we adopt here the definition proposed in [21]: 

Hoc{X I Y) = -\og2J2yeyPiy)^''^^^(^x p{x \ y) (2) 

We can now define the min-entropy leakage as loo = Hoo{X) — Hoo{X \ Y). The 
worst-case leakage is taken by maximising over all input distributions (recall that the 
input distribution models the attacker's side information): Coo — inaxp^(.) Ioo{X] Y). 
It has been proven in [16] that Coo is obtained at the uniform distribution, and that it 
is equal to the sum of the maxima of each column in the channel matrix, i.e.. Coo = 
Y,y^yn\a.yix(zxp{y \ x). 

Interpretation in terms of attacks. Min-entropy can be related to a model of adversary 
who is allowed to ask exactly one question of the form "is X = xl" (one-try attack). 
More precisely, Hoo(X) represents the (logarithm of the inverse of the) probability 
of success for this kind of attacks with the best strategy, which consists, of course, in 
choosing the x with the maximum probability. 

The conditional min-entropy Hoo {X \ Y) represents (the logarithm of the inverse 
of) the probability that the same kind of adversary succeeds in guessing the value of X 
a posteriori, i.e. after observing the result of Y. The complement of this probability is 
also known as probability of error or Bayes risk. Since in general X and Y are corre- 
lated, observing Y increases the probability of success. Indeed we can prove formally 
that Hoo{X I Y) < Hoo{X), with equality if and only if X and Y are independent. 
The min-entropy leakage Ioo{X] Y) = Hoo(X) — Hoo{X\Y) corresponds to the ratio 
between the probabilities of success a priori and a posteriori, which is a natural notion 
of leakage. Note that it is always the case that Ioo{X; Y) > 0, which seems desirable 
for a good notion of leakage. 

3 A model of utility and privacy for statistical databases 

In this section we present a model of statistical queries on databases, where noise is 
carefully added to protect privacy and, in general, the reported answer to a query does 
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not need to correspond to the real one. In this model, the notion of information leakage 
can be used to measure the amount of information that an attacker can learn about 
the database by posting queries and analysing their (reported) answers. Moreover, the 
model allows us to quantify the utility of the query, that is, how much information about 
the real answer can be obtained from the reported one. This model will serve as the basis 
for exploring the relation between differential privacy and information flow. 

We fix a finite set Ind ~ {1, 2, . . . , u} of u individuals participating in the database. 
In addition, we fix a finite set Val = {vi,V2, . . . , Vu}, representing the set of (v differ- 
ent) possible values for the sensitive attribute of each individual (e.g. disease-name in a 
medical database)' . Note that the absence of an individual from the database, if allowed, 
can be modeled with a special value in Val. As usual in the area of differential privacy 
[22], we model a database as a w-tuple D = {do, . . . , du-i} where each di G Val is 
the value of the corresponding individual. The set of all databases is A:" = VaZ". Two 
databases D, D' are adjacent, written D ^ D' iff they differ for the value of exactly 
one individual. 

Let /C be a randomized function from X to Z, 
where Z = Range{JC) (see Figure 1). This func- 
tion can be modeled by a channel with input and 
output alphabets X, Z respectively. This channel 
can be specified as usual by a matrix of condi- 
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tional probabilities Pz\x{'\')- We also denote by e-diff.priv. 

V ry 1 -LI ji- J mndomized fufictioTi 

X , Z the random variables modelmg the mput and ^ 
output of the channel. The definition of differen- 

tial privacy can be directly expressed as a property ^'S- 1-Randomized function /C as a 
of the channel: it satisfies e-differential privacy iff 

p{z\x) < e'^p{z\x') for all z G Z,x, x' G X with x ^ x' 

Intuitively, the correlation between X and Z measures how much information about 
the complete database the attacker can obtain by observing the reported answer We will 
refer to this correlation as the leakage of the channel, denoted by C{X, Z). In Section 4 
we discuss how this leakage can be quantified, using notions from information theory, 
and we study the behavior of the leakage for differentially private queries. 

We then introduce a random variable Y modeling the true answer to the query /, 
ranging over 3^ = Range{f). The correlation between Y and Z measures how much we 
can learn about the real answer from the reported one. We will refer to this correlation 
as the utility of the channel, denoted by UiY,Z). In Section 5 we discuss in detail 
how utility can be quantified, and we investigate how to construct a randomization 
mechanism, i.e. a way of adding noise to the query outputs, so that utility is maximized 
while preserving differential privacy. 

In practice, the randomization mechanism is often oblivious, meaning that the re- 
ported answer Z only depends on the real answer Y and not on the database X. In this 
case, the randomized function K,, seen as a channel, can be decomposed into two parts: 
a channel modeling the query /, and a channel modeling the oblivious randomization 



' In case there are several sensitive attributes in the database (e.g. skin color and presence of a 
certain medical condition), we can think of the elements of Val as tuples. 
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Fig. 2. Leakage and utility for oblivious mechanisms 



mechanism %. The definition of utihty in this case is simpUfied as it only depends on 
properties of the sub-channel correspondent to T-L. The leakage relating X and Y and the 
utility relating Y and Z for a decomposed randomized function are shown in Figure 2. 

Leakage about an individual. As already discussed, £,{X, Z) can be used to quantify 
the amount of information about the whole database that is leaked to the attacker How- 
ever, protecting the database as a whole is not the main goal of differential privacy. 
Indeed, some information is allowed by design to be revealed, otherwise the query 
would not be useful. Instead, differential privacy aims at protecting the value of each 
individual. Although C{X, Z) is a good measure of the overall privacy of the system, 
we might be interested in measuring how much information about a single individual is 
leaked. 

To quantify this leakage, we assume that the values of all other individuals are al- 
ready known, thus the only remaining information concerns the individual of interest. 
Then we define smaller channels, where only the information of a specific individual 
varies. Let D~ G Val"~^ be a (m — l)-tuple with the values of all individuals except 
the one of interest. We create a channel K-d- whose input alphabet is the set of all 
databases in which the u — 1 other individuals have the same values as in D~ . Intu- 
itively, the information leakage of this channel measures how much information about 
one particular individual the attacker can learn if the values of all others are known to 
he D~. This leakage is studied in Section 4.1. 

4 Leakage 

As discussed in the previous section, the correlation C{X, Z) between X and Z mea- 
sures the information that the attacker can learn about the database by observing the 
reported answers. In this section, we consider min-entropy leakage as a measure of this 
information, that is C{X, Z) ~ Ioo{X; Z). We then investigate bounds on information 
leakage imposed by differential privacy. These bounds hold for any side information of 
the attacker, modelled as a prior distribution on the inputs of the channel. 
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Our first result shows that the min-entropy leakage of a randomized function /C 
is bounded by a quantity depending on e, the numbers u, v of individuals and values 
respectively. We assume that v > 2. 

Theorem \. If K. provides e-differential privacy then for all input distributions, the 
min-entropy leakage associated to K, is bounded from above as follows: 

Ioo{X; Z)<u log2 ""-^—z 



Note 
u log2 
e 



that 



(ii-l+e 

has value when e 



this bound B{u,v,e) = 
is a continuous function in 
= 0, and converges 



to u log2 w as e approaches infinity. Figure 3 
shows the growth of B{u, v, e) along with e, 
for various fixed values of u and v. 

The following result shows that the 
bound B{u, v, e) is tight. 

Proposition 1. For every u, v, and e there 
exists a randomized function K, which pro- 
vides e-differential privacy and whose min- 
entropy leakage is Ioo{X;Z) = B(u,v,e) 
for the uniform input distribution. 




Fig. 3. Graphs of B{u, v, e) for u= 100 
and ?j=2 (lowest line), w = 10 (interme- 
diate line), and v = 100 (highest line), 
respectively. 



Example 1. Assume that we are interested in 
the eye color of a certain population Ind = 

{Alice, Bob}. Let Val = {a, b, c} where a stands for absent (i.e. the null value), 
b stands for blue, and c stands for coal (black). We can represent each dataset with a 
tuple dido, where do G Val represents the eye color of Alice (cases do ~ b and do = c), 
or that Alice is not in the dataset (case do = a). The value di provides the same kind 
of information for Bob. Note that w = 3. Fig 4(a) represents the set X of all possible 
datasets and its adjacency relation. We now construct the matrix with input X which 
provides e-differential privacy and has the highest min-entropy leakage. From the proof 
of Proposition 1, we know that each element of the matrix is of the form where a is 
the highest value in the matrix, i.e. a = , c) = jtt^' ^^'^ graph-distance 



(in Fig 4(a)) between (the dataset of) the row which contains such element and (the 
dataset of) the row with the highest value in the same column. Fig 4(b) illustrates this 
matrix, where, for the sake of readability, each value is represented simply by d. 

Note that the bound B{u, v, e) is guaranteed to be reached with the uniform input 
distribution. We know from the literature [16, 12] that the I^o of a given matrix has its 
maximum in correspondence of the uniform input distribution, although it may not be 
the only case. 

The construction of the matrix for Proposition 1 gives a square matrix of dimension 
v'^ X w". Often, however, the range of /C is fixed, as it is usually related to the possible 
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(a) The datasets and their ad- (b) The representation of the 

jacency relation matrix 

Fig. 4. Universe and highest min-entropy leakage matrix giving e-differential privacy for Exam- 
ple 1. 



answers to the query /. Hence it is natural to consider the scenario in which we are given 
a number r < f and want to consider only those /C's whose range has cardinality at 
most r. In this restricted setting, we could find a better bound than the one given by 
Theorem 1, as the following proposition shows. 

Proposition 2. Let IC be a randomized function and let r ~ \ Range (/C) \-IfJC provides 
e-differential privacy then for all input distributions, the min-entropy leakage associated 
to IC is bounded from above as follows: 

Io.{X;Z) < log2 ^ ^ 



(w - 1 + e^Y - {e^Y + (e^)" 
where £ = Llog„ r J . 

Note that this bound can be much smaller than the one provided by Theorem 1 . For 
instance, if r = v this bound becomes: 

v(e'Y 

l0g2 ■ 



V-1 + (e<^)« 



which for large values of u is much smaller than B{u, v, e). In particular, for v = 2 and 
u approaching infinity, this bound approaches 1, while B{u, v, e) approaches infinity. 

Let us clarify that there is no contradiction with the fact that the bound B{u,v,e) 
is strict: indeed it is strict when we are free to choose the range, but here we fix the 
dimension of the range. 

Finally, note that the above bounds do not hold in the opposite direction. Since 
min-entropy averages over all observations, low probability observations affect it only 
slightly. Thus, by introducing an observation with a negligible probability for one user, 
and zero probability for some other user, we could have a channel with arbitrarily low 
min-entropy leakage but which does not satisfy differential privacy for any e. 
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4.1 Measuring the leakage about an individual 

As discussed in Section 3, the main goal of differential privacy is not to protect infor- 
mation about the complete database, but about each individual. To capture the leakage 
about a certain individual, we start from a tuple G Val^^^ containing the given (and 
known) values of all other 1 individuals. Then we create a channel whose input Xjj^ 
ranges over all databases where the values of the other individuals are exactly those of 
D~ and only the value of the selected individual varies. Intuitively, Ioo{Xd- ; Z) mea- 
sures the leakage about the individual's value where all other values are known to be 
as in D~ . As all these databases are adjacent, differential privacy provides a stronger 
bound for this leakage. 

Theorem 2. If IC provides e-differentialprivsicy then for all G Val^^^ and for all 
input distributions, the min-entropy leakage about an individual is bounded from above 
as follows: 

/oo(Xc-;Z) <log2 

Note that this bound is stronger than the one of Theorem 1 . In particular, it depends 
only on e and not on u, w. 

5 Utility 

As discussed in Section 3, the utility of a randomized function /C is the correlation 
between the real answers Y for a query and the reported answers Z. In this section we 
analyze the utility U{Y, Z) using the classic notion of utility functions (see for instance 
[23]). 

For our analysis we assume an oblivious randomization mechanism. As discussed 
in Section 3, in this case the system can be decomposed into two channels, and the 
utility becomes a property of the channel associated to the randomization mechanism 
Ti which maps the real answer y G ^ into a reported answer z G Z according to given 
probability distributions Pz\y{-\-)- However, the user does not necessarily take z as her 
guess for the real answer, since she can use some Bayesian post-processing to maximize 
the probability of success, i.e. a right guess. Thus for each reported answer z the user can 
remap her guess to a value y' E y according to a remapping function p{z) : Z ^ y, 
that maximizes her expected gain. For each pair {y, y'), with y G y,y' = p{y), there is 
an associated value given by a gain (or utility) function g{y, y') that represents a score 
of how useful it is for the user to guess the value y' as the answer when the real answer 
is y. 

It is natural to define the global utility of the mechanism Ti. as the expected gain; 

u{Y,z) = Y,p{y)Y,p{y'\y)9{y.y') 0) 

V v' 

where p{y) is the prior probability of real answer y, and p{y'\y) is the probability of 
user guessing y' when the real answer is y. 
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We can derive the following characterization of the utility. We use to represent 
the probability distribution which has value 1 on x and elsewhere. 

U{Y, Z) = ^p(y) ^p(2/'|y)g(y,2/') (by (3)) 

V v' 

V v' \ z / 

= '^Piy)Yl [^Piz\y)^P(z){y') J 9{y,y') (by remaps = p{z)) 

V y' \ z I 

= ^p{y)^p{Av)^^p{z)[y')g[y.y') 

y z y' 

= P^y^ ^) 5Z ^pi^} (y'),9(y, y') 

y^z y' 

= Yp{y,z)g{y,p{z)) 



A very common utility function is the binary gain function, which is defined as 
9hin{y,y ) = 1 if y = y' and .9bin(y,y') = if y y'. The rationale behind this 
function is that, when the answer domain does not have a notion of distance, then the 
wrong answers are all equally bad. Hence the gain is total when we guess the exact 
answer, and is for all other guesses. Note that if the answer domain is equipped with 
a notion of distance, then the gain function could take into account the proximity of the 
reported answer to the real one, the idea being that a close answer, even if wrong, is 
better than a distant one. 

In this paper we do not assume a notion of distance, and we will focus on the binary 
case. The use of binary utihty functions in the context of differential privacy was also 
investigated in [20]^. 

By substituting g with gbin in the above formula we obtain: 

W(r,Z) = ^p(y,z)<5,(p(z)) (4) 

y,z 

which tells us that the expected utility is the greatest when p{z) = y is chosen to 
maximize p{y, z). Assuming that the user chooses such a maximizing remapping, we 
have: 



W(F,Z) = ;^maxp(y,z) 



(5) 



This corresponds to the converse of the Bayes risk, and it is closely related to the con- 
ditional min-entropy and to the min-entropy leakage: 

Hoo{Y\Z) = - log2 U{Y, Z) I^{Y- Z) = H^{X) + \og^ U{Y, Z) 



^ Instead of gain functions, [20] equivalently uses the dual notion of loss functions. 
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5.1 A bound on the utility 



In this section we show that the fact that /C provides e-differential privacy induces a 
bound on the utility. We start by extending the adjacency relation ^ from the datasets 
X to the answers y. Intuitively, the function / associated to the query determines a 
partition on the set of all databases {X, i.e. Val^), and we say that two classes are 
adjacent if they contain an adjacent pair More formally: 

Definition 2. Given y,y' G y, with y ^ y' , we say that y and y' are adjacent ( notation 
y ^ y'), iff there exist D, D' e VaP with D D' such that y = f{D) and y' = /(£>')• 

Since ^ is symmetric on databases, it is also symmetric on y, therefore also {y, ~) 
forms an undirected graph. 

Definitions. The distance dist between two elements y,y' G y is the length of the 
minimum path from y to y'. For a given natural number d, we define Border d [y) as the 
set of elements at distance d from y: 

Border d{y) = {y' \ dist{y,y') = d} 

We recall that a graph automorphism is a permutation of its vertices that preserves 
its edges. If <t is a permutation of S then an orbit of (t is a set of the form {(t*(s) | i G N} 
where s G S. A permutation has a single orbit iff {cr*(s) |i G N} = S' for all s G S. 

The next theorem provides a bound on the utility in the case in which {y, ■~) admits 
a graph automorphism with a single orbit. Note that this condition implies that the 
graph has a very regular structure; in particular, all nodes must have the same number 
of incident edges. Examples of such graphs are rings and cliques (but they are not the 
only cases). 

Tlieorem 3. Let T-Lbe a randomization mechanism for the randomized function K and 
the query f, and assume that K. provides e-differential privacy. Assume that {y, ~) 
admits a graph automorphism with a single orbit. Furthermore, assume that there exists 
a natural number c and an element y € y such that, for every natural number d > 0, 
either \Borderd{]j)\ = or \Borderd{y)\ > c. Then 



(e^)"(l - e«) + c(l - (e^)") 
where n is the maximum distance from y in y. 

The bound provided by the above theorem is strict in the sense that for every e 
and y there exist an adjacency relation ^ for which we can construct a randomization 
mechanism H that provides e-differential privacy and whose utility achieves the bound 
of Theorem 3. This randomization mechanism is therefore optimal, in the sense that it 
provides the maximum possible utility for the given e. Intuitively, the condition on ~ 
is that \Borderd{y)\ must be exactly c or for every d > 0. In the next section we 
will define formally such an optimal randomization mechanism, and give examples of 
queries that determine a relation ~ satisfying the condition. 
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5.2 Constructing an optimal randomization mechanism 

Assume f : X y, and consider the graph structure (y, ^) determined by /. Let 
n be the maximum distance between two nodes in the graph and let c be an integer 
We construct the matrix M of conditional probabilities associated to % as follows. For 
every column z € Z and every row y ^ y, define: 

Pz\Yiz\y) = ct/i^'^)'^ where d = distly, z) and a = ^ — ^— ^ ^ (6) 

(e=)"(l-e»)+c(l-(e')") 

The following theorem guarantees that the randomization mechanism T-L defined 
above is well defined and optimal, under certain conditions. 

Theorem 4. Let f : X y be a query and let e > 0. Assume that (3^, ^) admits 
a graph automorphism with a single orbit, and that there exists c such that, for every 
y € y and every natural number d > 0, either \Borderd{y)\ = or \Borderd{y)\ = c. 
Then, for such c, the definition in (6) determines a legal channel matrix for %, i.e., for 
each y (z y, Pz|y('|y) probability distribution. Furthermore, the composition K, 
of f and "H provides e-differential privacy. Finally, Ti is optimal in the sense that it 
maximizes utility when the distribution ofY is uniform. 

The conditions for the construction of the optimal matrix are strong, but there are 
some interesting cases in which they are satisfied. Depending on the degree of connec- 
tivity c, we can have several different cases whose extremes are: 

- (3^, ^) is a ring, i.e. every element has exactly two adjacent elements. This is sim- 
ilar to the case of the counting queries considered in [20], with the difference that 
our "counting" is in arithmetic modulo |3^|. 

- {y, ^) is a clique, i.e. every element has exactly |3^| — 1 adjacent elements. 

Remark 1. Note that when we have a ring with an even number of nodes the condi- 
tions of Theorem 4 are almost met, except that \Borderd{y)\ ~ 2 for d < n, and 
\ Border d{y)\ = 1 for d ~ n, where n is the maximum distance between two nodes in 
y. In this case, and if (e^)^ > 2, we can still construct a legal matrix by doubling the 
value of such elements. Namely, by defining 

Oi 

Pz\Y{z\y) = 2j^ \f dist{y,z) = n 
For all the other elements the definition remains as in (6). 

Remark 2. Note that our method can be applied also when the conditions of Theorem 4 
are not met: We can always add "artificial" adjacencies to the graph structure so to 
meet those conditions. Namely, for computing the distance in (6) we use, instead of 
{y, a structure [y, which satisfies the conditions of Theorem 4, and such that 
~ C rv^'. Naturally, the matrix constructed in this way provides e-differential privacy, 
but in general is not optimal. Of course, the smaller ~' is, the higher is the utility. 
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(a) 



Ml : truncated geometric mechanism (b) M2 : our mechanism 



In/Out 


A 


B 


C 


D 


E 


F 


A 


0.535 


0.060 


0.052 


0.046 


0.040 


0.267 


B 


0.465 


0.069 


0.060 


0.053 


0.046 


0.307 


C 


0.405 


0.060 


0.069 


0.060 


0.053 


0.353 


D 


0.353 


0.053 


0.060 


0.069 


0.060 


0.405 


E 


0.307 


0.046 


0.053 


0.060 


0.069 


0.465 


F 


0.267 


0.040 


0.046 


0.052 


0.060 


0.535 



In/Out 


A 


B 


C 


D 


E 


F 


A 


2/7 


1/7 


1/7 


1/7 


1/7 


1/7 


B 


1/7 


2/7 


1/7 


1/7 


1/7 


1/7 


C 


1/7 


1/7 


2/7 


1/7 


1/7 


1/7 


D 


1/7 


1/7 


1/7 


2/7 


1/7 


1/7 


E 


1/7 


1/7 


1/7 


1/7 


2/7 


1/7 


F 


1/7 


1/7 


1/7 


1/7 


1/7 


2/7 



Table 1. Mechanisms for the city with higher number of votes for a given candidate 



The matrices generated by our algorithm above can be very different, depending on 
the value of c. The next two examples illustrate queries that give rise to the clique and 
to the ring structures, and show the corresponding matrices. 

Example 2. Consider a database with electoral information where rows corresponds to 
voters. Let us assume, for simplicity, that each row contains only three fields: 

- ID: a unique (anonymized) identifier assigned to each voter; 

- CITY: the name of the city where the user voted; 

- CANDIDATE: the name of the candidate the user voted for. 

Consider the query "What is the city with the greatest number of votes for a given 
candidate ? " . For this query the binary function is a natural choice for the gain function: 
only the right city gives some gain, and any wrong answer is just as bad as any other. 

It is easy to see that every two answers are neighbors, i.e. the graph structure of the 
answers is a clique. 

Consider the case where CITY={A,B,C,D,E,F} and assume for simplicity that there 
is a unique answer for the query, i.e., there are no two cities with exactly the same num- 
ber of individuals voting for a given candidate. Table 1 shows two alternative mech- 
anisms providing e-differential privacy (with e = log 2). The first one, A/i, is based 
on the truncated geometric mechanism method used in [20] for counting queries (here 
extended to the case where every two answers are neighbors). The second mechanism, 
M2, is the one we propose in this paper. 

Taking the input distribution, i.e. the distribution on Y, as the uniform distribution, 
it is easy to see that U{Mi) = 0.2243 < 0.2857 = U{M2). Even for non-uniform dis- 
tributions, our mechanism still provides better utility. For instance, for p{A) ~ p{F) = 
1/10 andp(B) = p{C) = p{D) = P{E) = 1/5, we have W(Afi) = 0.2412 < 
0.2857 = U{M2). This is not too surprising: the Laplacian method and the geometric 
mechanism work very well when the domain of answers is provided with a metric and 
the utility function takes into account the proximity of the reported answer to the real 
one. It also works well when (3^, ^) has low connectivity, in particular in the cases of 
a ring and of a Une. But in this example, we are not in these cases, because we are 
considering binary gain functions and high connectivity. 

Example 3. Consider the same database as the previous example, but now assume a 
counting query of the form "What is the number of votes for candidate cand?". It 
is easy to see that each answer has at most two neighbors. More precisely, the graph 
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(a) A'h: truncated i-geom. mechanism 



In/Out 





1 


2 


3 


4 


5 





2/3 


1/6 


1/12 


1/24 


1/48 


1/48 


1 


1/3 


1/3 


1/6 


1/12 


1/24 


1/24 


2 


1/6 


1/6 


1/3 


1/6 


1/12 


1/12 


3 


1/12 


1/12 


1/6 


1/3 


1/6 


1/6 


4 


1/24 


1/24 


1/12 


1/6 


1/3 


1/3 


5 


1/48 


1/48 


1/24 


1/12 


1/6 


2/3 



Table 2. Mechanisms for 



(b) M2 : our mechanism 



In/Out 





1 


2 


3 


4 


5 





4/11 


2/11 


1/11 


1/11 


1/11 


2/11 


1 


2/11 


4/11 


2/11 


1/11 


1/11 


1/11 


2 


1/11 


2/11 


4/11 


2/11 


1/11 


1/11 


3 


1/11 


1/11 


2/11 


4/11 


2/11 


1/11 


4 


1/11 


1/11 


1/11 


2/11 


4/11 


2/11 


5 


2/11 


1/11 


1/11 


1/11 


2/11 


4/11 



counting query (5 voters) 



structure on the answers is a line. For illustration purposes, let us assume that only 5 
individuals have participated in the election. Table 2 shows two alternative mechanisms 
providing e-differential privacy (e = log 2): (a) the truncated geometric mechanism AIi 
proposed in [20] and (b) the mechanism A/2 that we propose, where c — 2 and n = 3. 
Note that in order to apply our method we have first to apply Remark 2 to transform 
the line into a ring, and then Remark 1 to handle the case of the elements at maximal 
distance from the diagonal. 

Le us consider the uniform prior distribution. We see that the utility of Mi is higher 
than the utility of A/2, in fact the first is 4/9 and the second is 4/11. This does not 
contradict our theorem, because our matrix is guaranteed to be optimal only in the case 
of a ring structure, not a line as we have in this example. If the structure were a ring, i.e. 
if the last row were adjacent to the first one, then A/i would not provide e-differential 
privacy. In case of a line as in this example, the truncated geometric mechanism has 
been proved optimal [20] . 



6 Related work 

As far as we know, the first work to investigate the relation between differential privacy 
and information-theoretic leakage /or an individual was [24]. In this work, a channel is 
relative to a given database x, and the channel inputs are all possible databases adjacent 
to X. Two bounds on leakage were presented, one for the Shannon entropy, and one for 
the min-entropy. The latter corresponds to Theorem 2 in this paper (note that [24] is an 
unpublished report). 

Barthe and Kopf [25] were the first to investigates the (more challenging) connec- 
tion between differential privacy and the min-entropy leakage/or the entire universe of 
possible databases. They consider only the hiding of the participation of individuals in 
a database, which corresponds to the case of w = 2 in our setting. They consider the 
"end-to-end differentially private mechanisms", which correspond to what we call K. in 
our paper, and propose, like we do, to interpret them as information-theoretic channels. 
They provide a bound for the leakage, but point out that it is not tight in general, and 
show that there cannot be a domain-independent bound, by proving that for any number 
of individual u the optimal bound must be at least a certain expression /(u, e). Finally, 
they show that the question of providing optimal upper bounds for the leakage of K. in 
terms of rational functions of e is decidable, and leave the actual function as an open 
question. In our work we used rather different techniques and found (independently) 
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the same function f{u, e) (the bound B{u, v, e) in Theorem 1 for v = 2), but we proved 
that f{u, e) is a bound, and therefore the optimal bound^. 

Clarkson and Schneider also considered differential privacy as a case study of their 
proposal for quantification of integrity [26]. There, the authors analyzed database pri- 
vacy conditions from the literature (such as differential privacy, fc-anonymity, and l- 
diversity) using their framework for utility quantification. In particular, they studied 
the relationship between differential privacy and a notion of leakage (which is different 
from ours - in particular their definition is based on Shannon entropy) and they provided 
a tight bound on leakage. 

Heusser and Malacaria [27] were among the first to explore the application of 
information-theoretic concepts to databases queries. They proposed to model database 
queries as programs, which allows for statical analysis of the information leaked by 
the query. However [27] did not attempt to relate information leakage to differential 
privacy. 

In [20] the authors aimed at obtaining optimal-utility randomization mechanisms 
while preserving differential privacy. The authors proposed adding noise to the output 
of the query according to the geometric mechanism. Their framework is very interesting 
because it provides us with a general definition of utility for a randomization mecha- 
nism AI that captures any possible side information and preference (defined as a loss 
function) the users of M may have. They proved that the geometric mechanism is opti- 
mal in the particular case of counting queries. Our results in Section 5 do not restrict to 
counting queries, however we only consider the case of binary loss function. 

7 Conclusion and future work 

An important question in statistical databases is how to deal with the trade-off between 
the privacy offered to the individuals participating in the database and the utility pro- 
vided by the answers to the queries. In this work we proposed a model integrating the 
notions of privacy and utility in the scenario where differential-privacy is applied. We 
derived a strict bound on the information leakage of a randomized function satisfying 
e-differential privacy and, in addition, we studied the utility of oblivious differential 
privacy mechanisms. We provided a way to optimize utility while guaranteeing differ- 
ential privacy, in the case where a binary gain function is used to measure the utility of 
the answer to a query. 

As future work, we plan to find bounds for more generic gain functions, possibly 
by using the Kantorovich metric to compare the a priori and a posteriori probability 
distributions on secrets. 
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Appendix 



Notation 

In the following we assume that A and B are random variables with carriers A and 
B, respectively. Let M be a channel matrix with input A and output B. We recall that 
the matrix M represents the conditional probabilities Pb\a{'\')- More precisely, the 
element of AI at the intersection of row a G A and column b £ Bis Ma^b = Ps|A(^|fl)- 
Note that if the matrix M and the input random variable A are given, then the output 
random variable B is completely determined by them, and we use the notation B{M, A) 
to represent this dependency. We also use (A) to represent the conditional min- 
entropy HaoiA\B{M, A)). Similarly, we use I^{A) to denote /oo(A; B{M, A)). 
We denote by M[l k] the matrix obtained by "collapsing" the column I into k, 

i.e. 

'M,^k + Ma j = k 
j = l 



M[l yfc],,. 



Mi J otherwise 



Given a partial function p : A ^ B, the image of A under p is p{A) = {p{a)\a G 

A, p{a) 7^ _L}, where ± stands for "undefined". 

In the proofs we need to use several indices, hence we typically use the letters 
h, k, I to range over rows and columns (usually i, h, I range over rows and j, k 

range over columns). Given a matrix M, we denote by max-' Af the maximum value of 

column j over all rows i, i.e. max-'M = max,; Mij . 



Proofs 

For the proofs, it will be useful to consider matrices with certain symmetries. In partic- 
ular, it will be useful to transform our matrices in square matrices having the property 
that the elements of the diagonal contain the maximum values of each column, and 
are all equal. This is the purpose of the following two lemmata: the first one trans- 
forms a matrix into a square matrix with all the column maxima in the diagonal, and 
the second makes all the elements of the diagonal equal. Both transformations preserve 
e-differential privacy and min-entropy leakage. 

Leakage 

In this part we prove the results about the bounds on min-entropy leakage. In the fol- 
lowing lemmata, we assume that AI has input A and output B, and that A has a uniform 
distribution. 

Lemma 1. Given an n x m channel matrix M with n < m, providing e-differential 
privacy for some e > 0, we can construct a square n x n channel matrix M' such that: 

1. M' provides e-differential privacy. 

2. Mj' j = max* M' /or all i G A, i.e. the diagonal contains the maximum values of 
the columns. 
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3. H^'iA)=H^iA). 

Proof. We first show that there exists an n x m matrix N and an injective total function 
p : B such that: 

- Ni^p(i) = maxP'-'^N for all i G A, 

- Nij = for all j G B\p{A) and all i G A. 

We iteratively construct p, N "column by column" via a sequence of approximating 
partial functions ps and matrices A^s (0 < s < rn). 

- Initial step (s = 0). 

Define po{i) = -L for a\li e A and A^o = M. 

- s*'' step (1 < s < to). 

Let j be the s-th column and let i G ^ be one of the rows containing the maximum 
value of column j in M, i.e. Mij = max-* A/. There are two cases: 

1. ps_i(i) = ±: we define 

ps = ps-i U{i^ j} 

2. ps-i{i) ~ k e B: we define 

As = Ps~l 

Ns^Ns-i[j ^ fc] 

Since the first step assigns j in ps and the second zeroes the column j in Ng, all 
unassigned columns ;B \ pm{A) must be zero in Nm- We finish the construction by 
taking p to be the same as pm after assigning to each unassigned row one of the columns 
in i3 \ pm{A) (there are enough such columns since n < to). We also take N = Nm- 
Note that by construction is a channel matrix. 

Thus we get a matrix N and a function p : A ^ B which, by construction, 
is injective and satisfies Ni^pi^i) = max''(''A^ for all i <=z A, and Ni^j ~ for all 
j G B\p{A) and all i & A. Furthermore, N provides e-differential privacy because 
each column is a linear combination of columns of M. It is also easy to see that 

max^ A^ = max-* A/, hence H^{A) = (A) (remember that A has the uni- 
form distribution). 

Finally, we create our claimed matrix M' from A^ as follows: first, we eUminate 
all columns in B\ p{A). Note that all these columns are zero so the resulting matrix 
is a proper channel matrix, provides differential privacy and has the same conditional 
min-entropy. Finally, we rearrange the columns according to p. Note that the order of 
the columns is irrelevant, any permutation represents the same conditional probabilities 
thus the same channel. The resulting matrix M' is n x n and has all maxima in the 
diagonal. □ 
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Lemma 2. Let M be a channel with input and output alphabets A = B = VaP\ and 
let ^ be the adjacency relation on Val^ defined in Section 3. Assume that the maximum 
value of each column is on the diagonal, that is Mi^i = max'Af /or all i ^ A. If M 
provides e-differential privacy then we can construct a new channel matrix M' such 
that: 

1. M' provides e-differential privacy; 

2. M[ j = M'l^ hf^^ all i, h Cz A i.e. all the elements of the diagonal are equal; 



Proof. Let I E Val" . Recall that dist{k, I) (distance between k and I) is the length of 
the minimum ^-path connecting k and / (Definition 3), i.e. the number of individuals in 
which k and I differ. Since A = B = Val^ we will use dist{-, ■) also between rows and 
columns. Recall also that Border d{h) = {fc G B\dist{h, k) = d}. For typographical 
reasons, in this proof we will use the notation Bh,d to represent Border d{h), and d{k, I) 
to represent dist{k, I). 

Let n = \ A\ = u". The matrix M' is given by 



We first show that this is a well defined channel matrix, namely J2keB ^^'h k ~ ^ ^'^^ 
all he A. We have 



Let A — {0, . . . , u}. Note that B = Udezi ^h,d7 and these sets are disjoint, so the 
summation over k E B can be split as follows 



3. = max*M' 

4. H^{A)=H^' 







as J2 



= 1, we obtain 
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and now the summations over j can be joined together 

ieAjeB 

We now show that the elements of the diagonal have the intended properties. First, we 
show that the elements of the diagonal are all the same. We have that Bi^d(h,h) = Bifi = 
{i} for all h E A, and therefore: 



Then, we show that they are the maxima for each column. Note that \Bi_d\ = {^{v — l)'^ 
which is independent of i. We have: 

^i'h I- = -rrr^^ r V V Mi 

n\Bh,d(h,k)\ PR 



<— Ma (Af has maxima in the diag.) 

1 \B, 



" t^A \^h,d{h,k)\ 

n ^-^ 

ieA 

It easily follows that max^A/' = max^A/ which implies that H^{A) = 

It remains to show that A/' provides e-differential privacy, namely that 

A/;l.fc < e^Ml^,j^ yh, h', keA-.hr^h' 
Since d{h, h') = 1, by the triangular inequality we derive: 

d(h\ k)-l< d{h, k) < d{h', k) + 1 
Thus, there are exactly 3 possible cases: 

1. d{h,k) = d{h',k). 

The result is immediate since Af^ j, = Af^, f.. 

2. d{h,k) = d{h',k) - 1. 
Define 

^'i.j = {/ e B^^d(i,j)+i\j' ^ j} 

Note that \Sij\ = {u — d{i,j)){v — 1) (i and j are equal in u — d{i,j) elements, 
and we can change any of them in w — 1 ways). The following holds: 

Mij < e^Mij' Vj' e Sij (diff. privacy) ^ 
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{u — d{i,j)){v — l)Mij < e'^ ^ Mij> (sum of the above) 

^ (u- d{h,k)){v - l)Mtj <e' ^ ^ Mij> (sum over j) 

Let d = d{h, k). Note that each j' £ is contained in exactly d + 1 different 

sets Si,j , j G Bi^d- So the right-hand side above sums all elements of Bi^d+i, d + 1 
times each. Thus we get 

{u -d)iv~l) ^^^3 <e'{d+l) M,,, (7) 

Finally, we have 

<e^ ,,, 7 ^7^7^^ E E ^^'J (from (7)) 



3. = d{h',k) + 1. 

Symmetrical to the case d{h, k) ~ d{h' , k) — 1. 



We are now ready to prove our first main result. 



□ 



Theorem 1. If /C provides e-differential privacy then the min-entropy leakage associ- 
ated to K. is bounded from above as follows: 

Proof. Let us assume, without loss of generality, that \X\ < |Z| (if this is not the 
case, then we add enough zero columns, i.e. columns containing only O's, so to match 
the number of rows. Note that adding zero columns does not change the min-entropy 
leakage). 

For our proof we need a square matrix with all column maxima on the diagonal, and 
all equal. We obtain such a matrix by transforming the matrix associated to IC as follows: 
first we apply Lemma 1 to it (with A = X and B = Z), and then we apply Lemma 2 
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to the result of Lemma 1. The final matrix M has size n x n, with n = \X\ = w", 
provides e-differential privacy, and for all rows /i we have that Ali^i = Mh,h and 
Mi = max*M. Furthermore, {X) is equal to the min-entropy leakage of /C. 

Let us denote by a the value of every element in the diagonal of M, i.e. a = Mi^i 
for every row i. Note that for every j E Border d{i) (i.e. every j at distance d from 
a given i) the value of A/^j is at least ^^Jy , hence Mij > j^y- Furthermore each 
element j at distance d from i can be obtained by changing the value of d individuals 
in the w-tuple representing i. We can choose those d individuals in (^) possible ways, 
and for each of these individuals we can change the value (with respect to the one in i) 
in w — 1 possible ways. Therefore \Borderd{i)\ ~ {v — I)'', and we obtain: 



Since each row represents a probability distribution, the elements of row i must sum up 
to 1. Hence: 





Now we apply some transformations: 




d=0 



Since a 




{v — 1 + e')" (binomial expansion), we obtain: 




(8) 



Therefore: 



(by definition) 




3 



log2 + log2 a 




(by (8) ) 



□ 
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The next proposition shows that the bound obtained in previous theorem is tight. 

Proposition 1. For every u, v, and e there exists a randomized function /C which pro- 
vides e-differential privacy and whose min-entropy leakage, for the uniform input dis- 
tribution, is Iao{X; Z) = -B(u, V, e). 

Proof. The adjacency relation in X determines a graph structure Gx- Set Z — X and 
define the matrix of K as follows: 

'Pk,{z\x) = — ^ ' ^ ' where d is the distance between x and z in Gx 

It is easy to see that -p-iQ^^x) is a probability distribution for every x, that K, provides 
e-differential privacy, and that lryo{X\ Z) ~ B{u, v, e). □ 

We consider now the case in which \Range{IC) \ is bounded by a number smaller 
than u". 

In the following when we have a random variable X, and a matrix M with row 
indices in A C. X, we will use the notations (X) and (X) to represent the 
conditional min-entropy and leakage obtained by adding "dummy raws" to M, namely 
rows that extend the input domain of the corresponding channel so to match the input 
X, but which do not contribute to the computation of (X). Note that it is easy to 
extend M this way: we only have to make sure that for each column j the value of each 
of these new rows is dominated by max^ A/'. 

We will also use the notation ^„ and ^£ to refer to the standard adjacency relations 
on Val" and Vaf, respectively. 

Lemma 3. Let JC be a randomized function with input X, where X = Val^"''^ , provid- 
ing e-differential privacy. Asssume that r — \Range{IC) \ = v^, for some £ < u. Let M 
be the matrix associated to JC. Then it is possible to build a square matrix M' of size 
X v^, with row and column indices in A X, and a binary relation ^'C A y. A 
such that {A^ ■~') is isomorphic to ( Vaf , ), and such that: 

L Ml J < (e^)"~'+'' Ml J for all k ^ A, where d is the ^' -distance between j 
and k. 

2. Ml j = ^i/c" 0.11 i,h € A i-e. elements of the diagonal are all equal 

3. Ml ^ = max'' M' for all i ^ A i-e. the diagonal contains the maximum values of 
the columns. 

4. H^'{X)^H^{X). 

Proof. We first apply a procedure similar to that of Lemma 1 to construct a square 
matrix of size x which has the maximum values of each column in the diago- 
nal. (In this case we construct an injection from the columns to rows containing their 
maximum value, and we eliminate the rows that at the end are not associated to any col- 
umn.) Then define ^' as the projection of ^„ on Val^ . Note that point 1 in this lemma 
is satisfied by this definition of Finally, apply the procedure in Lemma 2 (on the 
structure {A, ^')) to make all elements in the diagonal equal and maximal. Note that 
this procedure preserves the property in point 1, and conditional min-entropy. Hence 
H^'iX)^H^iX). □ 
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Proposition 2. Let /C be a randomized function and let r = \Range{IC)\. If /C provides 
e-differential privacy then the min-entropy leakage associated to /C is bounded from 
above as follows: 

r (e^)" 

Ioo{X;Z) < log2- / ,w , . ,w 

where £ = [log^ rj . 



Proof. Assume first that r is of the form v^. We transform the matrix AI associated to 
IC by applying Lemma 3, and let M' be the resulting matrix. Let us denote by a the 
value of every element in the diagonal of M', i.e. a = Ml ^ for every row i, and let us 
denote by Border' d{i) the border (Def 3) wrt Note that for every j G Border' d{i) 
we have that A/[, < j (e")"-^+'^, hence 

M , < 



*J — (pe\u-e+d 



Furthermore each element j at '-^'-distance d from i can be obtained by changing the 
value of d individuals in the £-tuple representing i (remember that ) is isomorphic 

to ( Vaf , sirrii)). We can choose those d individuals in (^) possible ways, and for each 
of these individuals we can change the value (with respect to the one in i) in — 1 
possible ways. Therefore 

\Border'S)\ = Q - 1)' 
Taking into account that for ^ we do not need to divide by (e^) "~^+'^, we obtain: 



a 



d=l ^ ^ ^ ^ j 



Since each row represents a probability distribution, the elements of row i must sum up 
to 1. Hence: 



a + 

d=l 



By performing some simple calculations, similar to those of the proof of Theorem 1, 
we obtain: 



iv-l + e^Y - (e^Y + {e'Y 
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Therefore: 



I^' {X) = i/oo {X) - H^' {X) (by definition) 

1 

= l0g2 U" + l0g2 X! "~ 

^ = 1 (10) 

= log2 v" + log2 — + log2(w^ a) 

^ l0g2 7 1 . ( e\i ,( e\u ) 

(u — 1 + e'^j* — [e'^y + (e'j" 

Consider now the case in which r is not of the form v^. Let i be the maximum 
integer such that < r, and let m = r — v^. We transform the matrix M associated 
to K. by collapsing the m columns with the smallest maxima into the m columns with 
highest maxima. Namely, let ji , 72 , • ■ • , jm the indices of the columns which have the 
smallest maxima values, i.e. max^* Af < max^ M for every column j ^ ji, j2, ■ ■ ■ , jm- 
Similarly, let fci , fc2 , . . . , fc™ be the indexes of the columns which have maxima values. 
Then, define 

N = M[ji ki][j2 ^ fc2] . . . [j,n km] 

Finally, eliminate the m zero-ed columns to obtain a matrix with exactly columns. It 
is easy to show that 

i^ix) < C{x)^ 

After transforming N into a matrix M' with the same min-entropy leakage as described 
in the first part of this proof, from (10) we conclude 



I^iX) < l!^ (X)- < log2 

□ 



We now turn our attention to the min-entropy leakage associated to an individual. 

Lemma 4. If a randomized function IC : A ^ B respects an e-ratio in the sense that 
P]c(b\a') < e"^ ■ pic(b\a") for all a' , a" £ A and b Cz B, then the min-entropy leakage 
from A to B is bounded by: 

Ioo{A-B) < elog^e 

Proof. For clarity reasons, in this proof we use the notation p{b\A = a) for the proba- 
bihty distributions pic{b\A = a) associated to /C. 
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-Hoo{A\B) = log2X;hP(^)maxQp(a|6) 
= 1082^6 maxa(p(6)p(a|6)) 
= log2 E6niaxa(p(a)p(6|a)) 
< log2l]hmaXa(p(a) p{b\a)) 
= log2l]he'p(6|a)maxa_p(a) 
= log2 {e" maxap{a) J2bPib\o-)) 
= log2 (e^maxap(a)) 
= log2 + logniaxap(a) 
= elog2 e - Hoo{A) 
Therefore: 



(by definition) 

(by the Bayes theorem) 

(by hypothesis on /C, for some fixed a) 

(by probability laws) 
(by definition) 



i/oo(A|B)>i7,o(A)-elog2e (11) 
This gives us a bound on the min-entropy leakage: 

{A; B) = H^{A) ~ Hoo{A\B) 

<elog2e (by (11)) 

□ 

Theorem 2. If /C provides e-dijferential privacy then for all e Var""' the mm- 
entropy leakage about an individual is bounded from above as follows: 

/oc(Xc-;^) <log2 

Proof. By construction, the elements of A/j- are all adjacent. Hence /C/j- respects an 
e-ratio. Thus we are allowed to apply Lemma 4 (with X = X]j- and JC ~ /C_d-), 
which gives immediately the intended result. □ 

Utility 

In this part we prove the results on utility. We start with a lemma which plays a role 
analogous to Lemma 2, but for a different kind of graph structure: in this case, we 
require the graph to have an automorphism with a single orbit. 

Lemma 5. Let M be the matrix of a channel with the same input and output alphabet 
A. Assume an adjacency relation ^ on A such that the graph {A, ~) has an automor- 
phism a with a single orbit. Assume that the maximum value of each column is on the 
diagonal, that is Mi^i = max"^ M for all i G A. If M provides e-differential privacy 
then we can construct a new channel matrix M' such that: 
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1. M' provides e-dijferential privacy; 

2. Ml, = M'^jjorall i,he A; 

3. = max'M'for all i G A; 

4. H^{A)^H^'iA). 

Proof. Let n ~ \A\. For every h^k ^ A let us define the elements of M' as: 

^ n—l 



n 



First we prove that M' provides e-differential privacy. For every pair h ^ / and every 
k: 



i=0 
ri-1 



< ^ e'^Mci(^i')^cr'{k) (by e-diff. privacy, for some / s.t. p{a^{h')) = k) 



1=0 



Now we prove that for every h. A/,' is a legal probability distribution. Remember that 

{cr*(fc)|0 <i<n — 1} = A since a has a single orbit. 



n 

fe=0 A:=0 1=0 



n—l n—l 

(/l),rT»(fc), 



n — l n—l 

1=0 fc=0 
n-1 ^ 

V-1 (since {CT*(fc)|0 < i < 71 - 1} = ^) 



i=0 
1 



Next we prove that the diagonal contains the maximum value of each column, i.e., for 

'fc,fc 



every fc, M'f, ^ = m&x'^M'. 



^k^k = -E^^'^'('=)^'^'W 

fc=0 
n— 1 

> - ^ M,.(,,),,,(fc) (since A/,, = max'^'(^)M) 



fe=0 
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Finally, we prove that I^' (A) I^{A). It is enough to prove that H^' (A) 



n-1 



^ n—1 n—1 

= - 5m M^Hh),aHh) (since {a'{h)\Q < i < n ~ 1} ^ A) 

h=0 i=0 
^ n — 1 

- HtiiA) (since M,. = max'^'COM) 



n-1 

n 



h=0 
tMi 



□ 



Theorem 3. Let H be a randomization mechanism for the randomized function K. and 
the query /, and assume that /C provides e-differential privacy. Assume that [y, ~) 
admits a graph automorphism with a single orbit. Furthermore, assume that there exists 
a natural number c and an element y G y such that, for every natural number d > 0, 
either \Borderd{y)\ = or \Borderd{y)\ > c. Then 



(e^)"(l - e') + c(l - (e^)") 
where n is the maximum distance from y in y. 



Proof. Consider the matrix M obtained by applying Lemma 1 to the matrix of H, and 
then Lemma 5 to the result of Lemma 1 . Let us call a the value of the elements in the 
diagonal of M. 

Let us take an element AIi,i = a. For each element j e Border d{Mi,i), the value of 
Mi J can be at most . Also, the elements of row i represent a probability distribution, 
so they sum up to 1 . Hence we obtain: 



a + ^|5orrferd(?/)|— ^ < 1 
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Now we perform some simple calculations: 



a + ^ \ Border d{y)\ 



d=l 



< 1 



< 1 



(since by hypothesis \Border{y, d)\ > c) 



a(e*^)" +ca ^(e')"-'' < {e' 



n-1 



a(e'^)" +ca (e^)* < (e*^)" (geometric progression sum) 



a(e')" + ca 
a < 



1 - (e^)" 
1 - 
(e")"(l - e<^) 



(e^)"(l - e<^) + c(l - (e<^)") 
Since ly{{Y, Z) = a, we conclude. 



□ 



Theorem 4. Let / ; A" — > ^ be a query and let e > 0. Assume that {y, ^) admits 
a graph automorphism with a single orbit, and that there exists c such that, for every 
y € y and every natural number d > 0, either \Borderd{y)\ = or \Borderd{y)\ = c. 
Then, for such c, the definition in (6) determines a legal channel matrix for %, i.e., 
for each y & y, Pz\y{'\v) is a probability distribution. Furthermore, the composition 
K. of f and % provides e-differential privacy. Finally, H is optimal in the sense that it 
maximizes utility when the distribution of Y is uniform. 

Proof. We follow a reasoning analogous to the proof of Theorem 3, but using | Border {y, d) \ 
c, to prove that 

U(Y Z) = ieTil~e^) 
^ ' ' (e^)"(l -e^) +c(l - (e^)") 

From the same theorem, we know that this is a maximum for the utility. □ 
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